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INTRODUCTION 


The  Frank  J.  Seiler  Research  Laboratory  at  the  U.S.  Air  Force  Academy 
initiated  a work  unit  in  May  of  1977  entitled  Studies  and  Applications  of 
Operations  Research.  The  objectives  of  this  work  unit  include: 

1.  developing  tractable  research  topics  through  interaction  with 
various  offices  throughout  the  Air  Force  and  Department  of  Defense; 

2.  acting  as  a catalyst  between  the  topic  sponsors  and  members  of  the 
Academy  faculty  by  keeping  them  aware  of  interesting  and  important 
research  topics  and  assisting  in  contacts  with  the  sponsoring 
offices; 

3.  conducting  in-house  research. 

Discussions  with  personnel  in  the  Registrar's  Office  at  the  Academy  in 
conjunction  with  objective  one  led  to  the  formulation  of  several  research 
topics,  one  of  which  is  the  subject  of  this  paper.  A preliminary  report1 
of  this  work  was  published  as  a technical  memorandum  in  November  1977, 
however,  the  substantive  portions  are  included  here. 

PROBLEM 

As  pilot  requirements  fluctuate  from  year  to  year,  so  too  must  pilot 
production  rates  for  Undergraduate  Pilot  Training  (UPT).  As  rates  are  cut 
back,  managers  must  become  more  selective  of  candidates  for  pilot  training. 
Thus,  it  behooves  the  managers  to  have  as  much  relevant  information  as 
possible  to  enlighten  their  selection  decision  for  Undergraduate  Pilot 
Training.  This  report  deals  specifically  with  the  problem  of  selecting 
from  among  Air  Force  Academy  graduates. 

OBJECTIVE 

The  objective  of  the  study  was  to  determine  if  existing  Air  Force 
Academy  cadet  data  could  be  useful  in  predicting  Undergraduate  Pilot  Training 
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performance.  If  one  is  able  to  predict  performance,  then  this  would  be 

2 3 

considered  when  selections  for  pilot  training  are  made.  Other  studies  ’ 
have  dealt  with  predicting  pilot  performance,  but  this  study  is  somewhat 
unique  in  that  it  addresses  only  AF  Academy  graduates  and  that  it  considers 
only  existing  data. 


DATA 

The  data  for  the  analysis  were  information  for  the  USAF  Academy  class 

of  1969  who  went  to  Undergraduate  Pilot  Training.  The  class  of  1969  was 

chosen  since  it  was  the  last  class  for  which  UPT  class  standing  information 

was  recorded.  Out  of  the  class  of  1969,  448  went  to  UPT.  Forty-six  (10.3%) 

did  not  complete  UPT  for  the  reasons  given  below. 

14  Flying  Deficient 

13  Medically  Deficient 

13  Self  Initiated  Elimination  (SIE) 

3 Manifestation  of  Anxiety  (MOA) 

2 Training  Deaths 
1 Death  Other 

APPROACH 

A two-part  approach  was  taken  to  determine  if  UPT  performance  was 
predictable  from  the  data.  One  part  was  to  examine  the  UPT  failures.  The 
other  part  was  to  apply  a linear  regression  program  to  the  data  for  those 
who  completed  UPT  to  determine  if  performance  could  be  predicted.  Those  not 
completing  UPT  had  no  UPT  performance  scores  (flying  grade,  academic  grade, 
class  standing)  so  they  could  not  be  included. 

RESULTS 

In  examining  UPT  failures  it  was  found  that  two  factors  had  significant 
influence  on  UPT  failure.  These  factors  were  T-41  training  status  and  cadet 
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medical  code.  The  T -41  training  status  was  whether  or  not  the  cadet  had 

completed  T -41  training.  Most  of  those  who  had  not  completed  T-41  training 

were  coded  as  not  entered.  The  cadet  medical  code  was  as  of  graduation  from 

the  Academy.  For  this  analysis  the  code  was  used  as  pilot  qualified  or 

waivered.  The  economics  of  the  T-41  program  and  the  waiver  of  vision 

4 

requirements  are  discussed  in  the  May  1978  "Interfaces."  The  article  draws 
on  analysis  conducted  at  the  USAF  Academy  in  the  latter  1960's.  It  indicates 
that  both  the  T-41  program  and  the  vision  waiver  are  easily  justified  from 
an  economic  standpoint.  This  is  mentioned  as  an  aside  here  since  the 
emphasis  of  this  paper  is  predicting  UPT  performance,  not  justifying  various 
programs.  Figure  1 shows  the  impact  of  the  two  variables  mentioned  above 
and  their  combinations  on  UPT  failure  rate. 

The  UPT  failure  rate  for  those  without  T-41  training  is  about  3 times 
the  rate  for  those  with  T-41  training.  The  UPT  failure  rate  for  those  with 
a medical  waiver  was  twice  the  rate  of  those  who  were  pilot  qualified. 

Combinations  follow  as  might  be  expected,  with  the  rate  for  those  without 
T-41  training  and  with  a waiver  being  nearly  5 times  that  of  those  with  the 
opposite  combination  of  factors.  The  reader  must  be  cautioned,  however, 
that  some  sample  sizes  were  small  as  indicated  by  the  chart.  However,  if 
one  allows  the  samples  one  can  claim  differences  in  proportions  with  high 
reliability.  The  levels  of  significance  are  presented  in  Table  1. 

A T-41  order  of  merit  score  was  computed  by  dividing  a cadet's  class 
standing  in  the  T-41  program  by  the  number  in  his  T-41  class  and  subtracting 
from  100%.  Thus,  a cadet  who  was  10th  from  the  top  of  a T-41  class  of  100 
would  receive  an  order  of  merit  score  of  90%.  The  average  T-41  order  of 
merit  score  was  computed  for  those  who  completed  UPT  and  for  those  who  did 
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not.  The  average  scores  were  53%  and  38%,  respectively.  This  further 
substantiates  that  UPT  performance  is  related  to  T-41  performance. 

One  might  suspect  that  most  of  those  who  do  not  have  T-41  training  and 
fail  UPT  would  fail  because  of  a flying  deficiency.  However,  as  shown  in 
Figure  2,  the  data  do  not  support  this.  Most  of  the  failures  among  those  who 
did  not  have  T-41  training  were  categorized  as  medical  deficiency. 

Similarly,  one  might  expect  that  most  of  those  who  are  medically 
waivered  and  fail  UPT  would  fail  because  of  a medical  deficiency.  Again, 
the  intuitive  is  not  supported  by  data.  In  fact.  Figure  3 indicates  that 
there  were  a higher  percentage  of  medical  deficiency  eliminations  among  those 
without  waivers  than  those  with  waivers.  Investigation  into  the  reasons  for 
the  medical  waivers  might  help  to  explain  this  and  might  identify  certain 
kinds  of  waivers  for  which  there  is  very  high  attrition. 

This  concluded  part  1 of  the  effort  with  indications  that  both  T-41 
training  and  medical  waiver  impact  UPT  performance. 

The  second  part  of  the  examination  consisted  of  using  1 inear-least- 
squares  regression  in  an  attempt  to  predict  UPT  performance  measures.  The 
variables  examined  are  presented  in  Table  2.  Obviously  many  of  the  predictor 
variables  are  correlated.  A cross  correlation  matrix  is  given  in  Table  3. 

All  order  of  merit  scores  were  converted  to  percent  scores  as  previously 
discussed  for  T-41  order  of  merit. 

Since  T-41  order  of  merit  was  not  available  when  tne  initial  runs  were 
made,  the  regression  did  not  include  it  when  attempting  to  predict  UPT 
academic  grade,  UPT  flying  grade,  and  UPT  pass/fail.  Later  on,  T-41  order 
of  merit  was  run  against  the  variables  individually. 
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The  UPT  academic  grade  was  a percentage  score  attained  for  the  academic 

portion  of  UPT.  The  scores  were  ajl  very  high  with  the  average  score  being 

93.9%.  The  highest  correlation  with  the  UPT  academic  grade  was  academic 

order  of  merit  at  graduation  from  the  Academy.  That  correlation,  though 

significant,  was  disappointing  with  an  r (simple  correlation  coefficient 

squared)  of  only  .30.  Even  with  all  variables  in  the  equation  (except  T-41 

o 

order  of  merit),  the  R (multiple  correlation  coefficient  squared)  was  only 
.33.  An  r of  .15  was  obtained  with  T-41  order  or  merit  alone.  Thus, 
although  it  appears  that  significant  correlation  exists,  it  is  not  sufficient 
to  use  for  prediction  purposes. 

The  UPT  flying  grades  were  also  quite  high  with  an  average  of  92.2%. 

There  was  complete  lack  of  correlation,  however,  with  an  ' of  .06  with  all 

variables  (except  T-41  order  of  merit)  in  the  equation. 

Equally  poor  results  were  obtained  in  attempting  to  predict  UPT  pass/ 

2 

fail  (a  0/1  variable).  An  R of  .07  indicating  no  correlation  was  obtained 
with  all  variables  (except  T-41  order  of  merit)  in  the  equation.  Although 
the  two  variables  (medical  code  and  T-41  training  status)  were  shown  to  have 
significant  impact  on  UPT  performance,  their  correlations  with  UPT  pass/fail 
produced  r of  only  .04  and  .02,  respectively.  This  result  was  not 
unexpected  since  many  with  waivers  and/or  no  T-41  training  completed 
UPT. 

It  appeared  at  this  point  in  the  study  that  regression  results  might  be 
improved  if  a better  measure  of  UPT  performance  were  available.  Ther-.cre, 
UPT  order  of  merit  for  the  class  of  ‘69  was  obtained  from  Air  Training 
Command.  Since  the  class  of  '69  was  divided  into  several  UPT  classes,  an 
order  of  merit  score  was  calculated  as  before.  However,  in  this  case  UPT 
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classes  were  mixed  with  other  than  Academy  graduates  so  order  of  merit 

scores  were  with  respect  to  total  UPT  classes.  This  should  not  have  a 

significant  effect  on  the  analysis.  The  average  UPT  order  of  merit  score 

for  the  class  of  '69  was  56.6%  so  it  appears  that  as  a class  they  did  slightly 

better  in  UPT  than  those  from  sources  other  than  the  Academy.  The  results  of 

regression  against  UPT  order  of  merit  are  presented  in  Table  4.  Although 

2 

improved  results  were  obtained,  the  R of  .34  is  not  high  enough  to  support 

reliable  prediction  of  UPT  performance.  Note  the  correlation  with  T-41  order 

2 

of  merit  produced  in  r of  only  .14. 

OBSERVATIONS 

The  results  indicate  that  both  medical  status  and  T-41  training 
influence  performance  in  UPT.  However,  even  with  the  most  undesirable 
combination  of  the  two  factors,  2/3  complete  UPT.  Thus,  while  the  factors 
are  useful  as  indicators,  they  are  not  useful  as  predictors. 

The  regression  analysis  showed  some  correlation  between  predictors  and 
UPT  performance  measures  but  no  factor  or  combination  of  factors  could 
reliably  predict  UPT  performance. 
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FIGURE  1 

PERCENTAGE  OF  UNDERGRADUATE  PILOT  TRAINING  FAILURES 

VERSUS 

Ml  TRAINING  STATUS  AND  MEDICAL  STATUS 


SMALL  SAMPLE  SIZE  (6/22  = 27.3%) 
SMALL  SAMPLE  SIZE  (4/12  = 33.3%) 
SMALL  SAMPLE  SIZE  (2/10  = 20.0%) 
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24/158  = 15.2%  FAILURES  22/290  = 7.6%  FAILURES 


TABLE  1 

LEVEL  OF  SIGNIFICANCE  FOR  DIFFERENCE  IN  PROPORTIONS 
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TABLE  3 

CROSS  CORRELATION  MATRIX 


MCD 

MCND 

VAPT 

MAPT 

ECOM 

MACH 

GPA 

ACOM 

LDOM 

GROM 

T41T 

T410M 

MCD 

1.00 

.38 

.08 

.03 

.08 

.09 

.13 

-.11 

-.02 

cx> 

o 

1 

1 

o 

.05 

MCND 

1.00 

.06 

.08 

.07 

.06 

.11 

-.10 

-.01 

-.09 

-.10 

-.04 

VAPT 

1.00 

.30 

.59 

.33 

.32 

-.33 

.09 

-.23 

.09 

.13 

MAPT 

, 

1.00 

.26 

.67 

.37 

-.36 

.05 

-.28 

.06 

.17 

ECOM 

1.00 

.35 

.29 

-.28 

-.06 

-.26 

.04 

.02 

MACH 

1.00 

.37 

-.36 

.08 

-.29 

.13 

.13 

GPA 

1.00 

-.98 

-.16 

-.88 

.06 

.34 

ACOM 

1.00 

.14 

.89 

.07 

-.31 

LDOM 

1.00 

.50 

.02 

-.06 

GROM 

1.00 

-.05 

-.29 

T41T 

1.00 

X 

T410M 

— 



1.00 

VARIABLES 

MCD Medical  Code  Cadet  (at  graduation) 

MCND— -Medical  Code  Candidate  (prior  to  entry) 

VAPT — Verbal  Aptitude  (College  Entrance  Exam  Board  Score  (CEEBS)) 
MAPT— Math  Aptitude  (CEEBS) 

ECOM — English  Composite  (CEEBS) 

MACH — Math  Achievement  (CEEBS) 

GPA Grade  Point  Average 

ACOM — Academic  Order  of  Merit 
LDOM---Leadership  Order  of  Merit 
GROM — Graduate  Order  of  Merit 
T41T— Cadet  T-41  Training  (yes/no) 

T410M--Cadet  T-41  Training  Order  of  Merit 
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MULTIPLE  CORRELATION  COEFFICIENT  SQUARED 


